On the Convergence of Markovian Stochastic Algorithms with Rapidly Decreasing Ergodicity Rates

We analyse the convergence of stochastic algorithms with Markovian noise when the ergodicity of the Markov chain governing the noise rapidly decreases as the control parameter tends to innnity. In such a case, there may be a positive probabilityof divergence of the algorithm in the classic Robbins-Monro form. We provide modiications of the algorithm which ensure convergence. Moreover, we analyse the asymptotic behaviour of these algorithms and state a diiusion approximation theorem. 1. Introduction Stochastic algorithms of Robbins-Monro type with Markovian noise form a category of processes for which almost sure convergence cannot be obtained in general. The reason is that the ergodicity of the Markov chain governing the noise may decrease when the control parameter tends to innnity, and trap the algorithm within an exploding regime. In this paper, we study rigorously a natural strategy in which more time is spent for estimating the variations of the control parameter for large values of this parameter. In particular, we give conditions under which this strategy converges.